Connectionist Temporal Modeling for Weakly Supervised Action Labeling

We propose a weakly-supervised framework for action labeling in video, where only the order of occurring actions is required during training time. The key challenge is that the per-frame alignments between the input (video) and label (action) sequences are unknown during training. We address this by introducing the Extended Connectionist Temporal Classification (ECTC) framework to efficiently evaluate all possible alignments via dynamic programming and explicitly enforce their consistency with frame-to-frame visual similarities. This protects the model from distractions of visually inconsistent or degenerated alignments without the need of temporal supervision. We further extend our framework to the semi-supervised case when a few frames are sparsely annotated in a video. With less than 1% of labeled frames per video, our method is able to outperform existing semi-supervised approaches and achieve comparable performance to that of fully supervised approaches.Comment: To appear in ECCV 201

Learning to Extract Motion from Videos in Convolutional Neural Networks

This paper shows how to extract dense optical flow from videos with a convolutional neural network (CNN). The proposed model constitutes a potential building block for deeper architectures to allow using motion without resorting to an external algorithm, \eg for recognition in videos. We derive our network architecture from signal processing principles to provide desired invariances to image contrast, phase and texture. We constrain weights within the network to enforce strict rotation invariance and substantially reduce the number of parameters to learn. We demonstrate end-to-end training on only 8 sequences of the Middlebury dataset, orders of magnitude less than competing CNN-based motion estimation methods, and obtain comparable performance to classical methods on the Middlebury benchmark. Importantly, our method outputs a distributed representation of motion that allows representing multiple, transparent motions, and dynamic textures. Our contributions on network design and rotation invariance offer insights nonspecific to motion estimation

Basic tasks of sentiment analysis

Subjectivity detection is the task of identifying objective and subjective sentences. Objective sentences are those which do not exhibit any sentiment. So, it is desired for a sentiment analysis engine to find and separate the objective sentences for further analysis, e.g., polarity detection. In subjective sentences, opinions can often be expressed on one or multiple topics. Aspect extraction is a subtask of sentiment analysis that consists in identifying opinion targets in opinionated text, i.e., in detecting the specific aspects of a product or service the opinion holder is either praising or complaining about

C. elegans feed yolk to their young in a form of primitive lactation

The nematode Caenorhabditis elegans exhibits rapid senescence that is promoted by the insulin/IGF-1 signalling (IIS) pathway via regulated processes that are poorly understood. IIS also promotes production of yolk for egg provisioning, which in post-reproductive animals continues in an apparently futile fashion, supported by destructive repurposing of intestinal biomass that contributes to senescence. Here we show that post-reproductive mothers vent yolk which can be consumed by larvae and promotes their growth. This implies that later yolk production is not futile; instead vented yolk functions similarly to milk. Moreover, yolk venting is promoted by IIS. These findings suggest that a self-destructive, lactation-like process effects resource transfer from postreproductive C. elegans mothers to offspring, in a fashion reminiscent of semelparous organisms that reproduce in a single, suicidal burst. That this process is promoted by IIS provides insights into how and why IIS shortens lifespan in C. elegans

Calcium Binds to Transthyretin with Low Affinity

The plasma protein transthyretin (TTR), a transporter for thyroid hormones and retinol in plasma and cerebrospinal fluid, is responsible for the second most common type of systemic (ATTR) amyloidosis either in its wild type form or as a result of destabilizing genetic mutations that increase its aggregation propensity. The association between free calcium ions (Ca2+) and TTR is still debated, although recent work seems to suggest that calcium induces structural destabilization of TTR and promotes its aggregation at non-physiological low pH in vitro. We apply high-resolution NMR spectroscopy to investigate calcium binding to TTR showing the formation of labile interactions, which leave the native structure of TTR substantially unaltered. The effect of calcium binding on TTR-enhanced aggregation is also assessed at physiological pH through the mechano-enzymatic mechanism. Our results indicate that, even if the binding is weak, about 7% of TTR is likely to be Ca2+-bound in vivo and therefore more aggregation prone as we have shown that this interaction is able to increase the protein susceptibility to the proteolytic cleavage that leads to aggregation at physiological pH. These events, even if involving a minority of circulating TTR, may be relevant for ATTR, a pathology that takes several decades to develop

Cognitive impairment in Parkinson's disease: impact on quality of life of carers.

BACKGROUND: The quality of life (QoL) of informal caregivers of people with Parkinson's disease (PD) (PwP) can be affected by the caring role. Because of cognitive symptoms and diminished activities of daily living, in addition to the management of motor symptoms, carers of PwP and cognitive impairment may experience increased levels of burden and poorer QoL compared with carers of PwP without cognitive impairment. This study aimed to investigate the impact of cognitive impairment in PD upon QoL of carers. METHODS: Approximately 36 months after diagnosis, 66 dyadic couples of PwP and carers completed assessments. PwP completed a schedule of neuropsychological assessments and QoL measures; carers of PwP completed demographic questionnaires and assessments of QoL. Factor scores of attention, memory/executive function and global cognition, as derived by principal component analysis, were used to evaluate cognitive domains. RESULTS: Hierarchical regression analysis found lower Montreal Cognitive Assessment was a significant independent predictor of poorer carer QoL, in addition to number of hours spent caregiving, carer depression and PD motor severity. Attentional deficits accounted for the largest proportion of variance of carer QoL. Carers of PwP and dementia (n = 9) had significantly poorer QoL scores compared with PwP and mild cognitive impairment (n = 18) or normal cognition (n = 39) carers (p < 0.01). CONCLUSIONS: Attentional deficits were the strongest predictor of carer QoL compared with other cognitive predictors. Carers for those with PD dementia reported the poorest QoL. Interventions such as respite or cognitive behavioural therapy to improve mood and self-efficacy in carers may improve carer QoL. © 2016 The Authors. International Journal of Geriatric Psychiatry published by John Wiley & Sons, Ltd

Asynchronous Interaction Aggregation for Action Detection

Understanding interaction is an essential part of video action detection. We propose the Asynchronous Interaction Aggregation network (AIA) that leverages different interactions to boost action detection. There are two key designs in it: one is the Interaction Aggregation structure (IA) adopting a uniform paradigm to model and integrate multiple types of interaction; the other is the Asynchronous Memory Update algorithm (AMU) that enables us to achieve better performance by modeling very long-term interaction dynamically without huge computation cost. We provide empirical evidence to show that our network can gain notable accuracy from the integrative interactions and is easy to train end-to-end. Our method reports the new state-of-the-art performance on AVA dataset, with 3.7 mAP gain (12.6% relative improvement) on validation split comparing to our strong baseline. The results on dataset UCF101-24 and EPIC-Kitchens further illustrate the effectiveness of our approach. Source code will be made public at: https://github.com/MVIG-SJTU/AlphAction

Spatio-Temporal Fusion Networks for Action Recognition

The video based CNN works have focused on effective ways to fuse appearance and motion networks, but they typically lack utilizing temporal information over video frames. In this work, we present a novel spatio-temporal fusion network (STFN) that integrates temporal dynamics of appearance and motion information from entire videos. The captured temporal dynamic information is then aggregated for a better video level representation and learned via end-to-end training. The spatio-temporal fusion network consists of two set of Residual Inception blocks that extract temporal dynamics and a fusion connection for appearance and motion features. The benefits of STFN are: (a) it captures local and global temporal dynamics of complementary data to learn video-wide information; and (b) it is applicable to any network for video classification to boost performance. We explore a variety of design choices for STFN and verify how the network performance is varied with the ablation studies. We perform experiments on two challenging human activity datasets, UCF101 and HMDB51, and achieve the state-of-the-art results with the best network

Orientation cues for high-flying nocturnal insect migrants: do turbulence-induced temperature and velocity fluctuations indicate the mean wind flow?

Migratory insects flying at high altitude at night often show a degree of common alignment, sometimes with quite small angular dispersions around the mean. The observed orientation directions are often close to the downwind direction and this would seemingly be adaptive in that large insects could add their self-propelled speed to the wind speed, thus maximising their displacement in a given time. There are increasing indications that high-altitude orientation may be maintained by some intrinsic property of the wind rather than by visual perception of relative ground movement. Therefore, we first examined whether migrating insects could deduce the mean wind direction from the turbulent fluctuations in temperature. Within the atmospheric boundary-layer, temperature records show characteristic ramp-cliff structures, and insects flying downwind would move through these ramps whilst those flying crosswind would not. However, analysis of vertical-looking radar data on the common orientations of nocturnally migrating insects in the UK produced no evidence that the migrants actually use temperature ramps as orientation cues. This suggests that insects rely on turbulent velocity and acceleration cues, and refocuses attention on how these can be detected, especially as small-scale turbulence is usually held to be directionally invariant (isotropic). In the second part of the paper we present a theoretical analysis and simulations showing that velocity fluctuations and accelerations felt by an insect are predicted to be anisotropic even when the small-scale turbulence (measured at a fixed point or along the trajectory of a fluid-particle) is isotropic. Our results thus provide further evidence that insects do indeed use turbulent velocity and acceleration cues as indicators of the mean wind direction